Identification of Paintings in Camera-Phone Images

This paper presents the algorithm developed, for the EE 368 course project, to identify paintings in query images taken with a camera-phone in the European Gallery of the Cantor Arts Center. The image processing algorithm first extracts the painting of interest, in the center of the photo. It uses edge detection in saturation space, followed by iterative Radon transforms and region removal to find the smallest bounding box enclosing the painting. Using the smallest bounding box, the algorithm computes and performs a projective transform to extract and rectify the painting in the query image. Feature identification is performed on the rectified image using extrema in a difference-of-Gaussians pyramid. At each resulting feature location, feature descriptors are computed using histograms of the gradient distribution among neighboring pixels, without rotation. The resulting features are correlated with a feature database of paintings with aspect ratios similar to that of the painting in the query image. A program to produce the feature database was also developed. This program selects a small set of unique, but repeatable, features from the many available for each painting in the gallery. The image processing algorithm correctly identifies paintings in all 99 training images provided by the EE368 teaching staff. It performs well on the additional 93 test images taken by the group, in difficult configurations, to stress test the algorithm. Details and sensitivities of the algorithm are discussed.